Image processing apparatus using the difference among scaled images as a layered image and method thereof

ABSTRACT

The present invention is to provide an image processing apparatus using the difference among scaled images as a layered image and a method thereof, which utilize the Gaussian and Laplacian pyramid theory to convert an original image into a plurality of scaled images of different scales, and the difference among scaled images of two adjacent different scales as a layered image of the corresponding layer, so that the edge and line characteristics of a scene of the original image for each layered image can be displayed in different levels sequentially from a clear level to a vague level, and provide a layered image display interface and an image characteristic editing interface for users to examine each layered image through the layered image display interface and edit or perform special effect to each layered image, so as to simulate different visual effects based on different vision models.

FIELD OF THE INVENTION

The present invention relates to an image processing apparatus andmethod, more particularly to an image processing apparatus and method byusing the difference among scaled images as a layered image for enablinga user to edit or perform special effect to the edge and linecharacteristics of the layered images of a scene of an original image,so as to simulate different visual effects based on different visionmodels.

BACKGROUND OF THE INVENTION

In general, an image processing software usually comes with a layerprocessing function that allows each image to be resided at a layer allthe time as shown in FIG. 1, and thus users usually need to create alayer 111, 121 or add a layer before processing (including editing orcomposing) an image 110, 120 in a file, and a canvas 130 is situated atthe lowest layer of all layers 111, 121 but the canvas 130 is not alayer by itself. Therefore, the layers 111, 121 are similar to a stackof transparent films, and the layers 111, 121 can help a user toorganize the images 110, 120 in a file, so that the user can edit theimage 110 on the layer 111 without affecting the image 120 on the otherlayer 121. If the layer 111 contains no image 110, then the user will beable to see the image 120 on the layer 121 through the layer 111. Inaddition, a user can view the stack sequence of the layers and images ona layer display panel provided by different image processing software,and such stack sequence is also the sequence of images appeared in thedocument. In general, various different image processing software stackthe layers according to such sequence, and the lately produced layer isplaced at the top of the stack, and the stack sequence determines how tostack an image of a layer on an image of another layer, such that userscan rearrange the sequence of layers and images through a controlinterface of the image processing software to change the content of theimages in a file.

Referring to FIG. 2 for an image processing software called “Photoshop”available in the market, a user can create two images 210, 220 on twoseparate layers from a layer display panel provided by the imageprocessing software, if the user wants to compose two images 210, 220. Acomposed image 310 is created as shown in FIG. 3, after the size andposition of each image 210, 220 are adjusted according to the user'srequirements. Referring to FIG. 4, a user can click on an [Add Mask]button 450 to create a mask 440 for a first image 410 on the first layer411, if the user wants to edit the first image 410 on the layer displaypanel 400. In FIG. 5, a brush tool 471 is selected from a tool menu 470.In FIGS. 5 and 6, the selected brush tool 471 is used to paint the mask440, such that the dark color position of a second image 420 at a secondlayer 421 corresponding to the mask 440 is set to a black color. If theuser selects to apply a gentle pressure of the brush tool 471, the userneeds to adjust the transparency of the black color, such that the image410 (or foreground) at the first layer 411 is merged with the secondimage 420 (or background) at the second layer 421 to form a composedimage 430 with the best composition effect.

From the description above, the image processing software available inthe market can use the concept and technique of a layer to provide atool for editing the image on each layer, or rearranging the imagesequence of each layer, and can use the image composition technology tosimulate a digital dark room, for editing and composing images, butthese image processing software cannot show the effect of differentvision models for the image.

As LMS is a color space used for indicating the response of three kindsof cones of human eyes, which refer to the sensitivity of color lightswith a long wavelength, a medium wavelength and a short wavelengthrespectively, and the cross-section of a human retinal-cortical systemincludes complicated neural links. Only the LMS cone is known ingeneral, but the profound structure of the retinal-cortical system stillincludes tree-structured constructions of cells strains, and the root ofthese constructions link several cones together to form a so-called“ganglion cell”, for making several receptive fields. Althoughneurophysists already have relatively high understanding on the visualimaging method of a retinal-cortical system of human eyes, yet thisunderstanding is limited to an edge enhancement effect only, since theganglion cells of different sizes are distributed increasingly denserfrom a fovea to the peripheral areas of a cornea, and thus the basicvisual imaging principle decreases the vision from the center of thevisual line to the peripheral areas. While human eyes are viewing ascene, an image sensed by a fixation point or a perceptual field of aneye cannot be the same as a camera or camcorder as shown in FIG. 7 fortreating each position on the image 510 the same way. In fact, humaneyes as shown in FIG. 8 only perceive an area (such as the cross area)interested to the eyes for a longer time, but the fixation point orperceptive field varies and the level of fixation is different. As aresult, an image 530 with the effect of a different vision model can beproduced in an imaging area of a human brain as shown in FIG. 9.

In recent years, unsharp-masking (USM) has been used extensively indifferent traditional image processing software and provided for usersto perform special effects for an image. Regardless of the computationprogram used by these image processing software, the core technologyuses the Laplacian of Gaussian edge enhancement technology and conceptto simulate the effects of a receptive field of a human vision, but thecomputation carries out a single-level processing for the image only,and it cannot show the effect of different vision models for the image.

SUMMARY OF THE INVENTION

In view of the foregoing shortcomings of the prior art, the inventor ofthe present invention based on years of experience in the relatedindustry to conduct extensive researches and experiments, and finallydeveloped an image processing apparatus using the difference amongscaled images as a layered image and its method in accordance with thepresent invention.

It is a primary objective of the present invention to provide an imageprocessing apparatus using the difference among scaled images as alayered image and its method to simulate an image with an effect ofdifferent vision models of the same scene in an imaging area of a humanbrain according to the fixation point, perceptive field or level offixation of a human eye that views the scene, while a user is editingthe image.

Another objective of the present invention is to use the Gaussian andLaplacian pyramid theory to convert an original image into a pluralityof scaled images of different scales, and the difference among scaledimages of two adjacent different scales as a layered image of thecorresponding layer, so that the edge and line characteristics of ascene of the original image for each layered image can be displayed indifferent levels sequentially from a clear level to a vague level, andprovide a layered image display interface and an image characteristicediting interface for users to examine each layered image through thelayered image display interface. According to actual requirements, theimage characteristic editing interface is used for editing or performinganother special effect for each layered image, so as to simulatedifferent visual effects based on different vision models.

A further objective of the present invention is to provide a layeredimage editing interface, and the layered image editing interfaceincludes a layered image editing window for displaying each editinglayered image or a composed image composed of the layered images, andits periphery has at least one layered image characteristic adjustingbutton for users to click to adjust the contrast or Gaussian variance ofthe layered image; at least one size adjusting button for users to clickto zoom in or out the layered image or composed image; and at least oneimage composition switch button for users to click to switch the layeredimage editing window and select each layered image or composed image, sothat a user can click the image composition switch button to browse eachediting layered image or composed image.

Another further objective of the present invention is to provide animage output interface, for integrating the edited layered image into anew image, and converting the new image into an image file having aspecific format for saving, displaying or printing the image file.

To make it easier for our examiner to understand the objective,technical characteristics and effects of the present invention,preferred embodiment will be described with accompanying drawings asfollows:

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of images and layers in various differentimage files of a prior art;

FIG. 2 is a schematic view of two images;

FIG. 3 is a schematic view of using Photoshop image processing softwareto compose two images as depicted in FIG. 2 into one image;

FIG. 4 is a schematic view of using Photoshop image processing softwareto add a mask operation to the composed image as depicted in FIG. 3;

FIG. 5 is a schematic view of a tool menu of Photoshop image processingsoftware;

FIG. 6 is a schematic view of using Photoshop image processing softwareto add a new mask to the image as depicted in FIG. 3 for a composition;

FIG. 7 is a schematic view of a scene captured by a camera or a videocamera;

FIG. 8 is a schematic view of an area (such as a cross area) of acertain scene attracted to human eyes;

FIG. 9 is a schematic view of simulating an image with an areainterested to and viewed by a human eye as shown in FIG. 8 for a longertime;

FIG. 10 is a schematic view of simulating five visual model spacesaccording to human vision sensing experience;

FIG. 11 is a schematic view of five levels of visual model spaces asshown in FIG. 10;

FIG. 12 is a schematic view of converting an original image into aplurality of scaled images of different scales by using the Gaussian andLaplacian pyramid theory;

FIG. 13 is a schematic view of using an image difference among scaledimages of two adjacent different scales to compute a layered imagecorresponding to a layer;

FIG. 14 is a schematic view of a structure of an image processingapparatus in accordance with a preferred embodiment of the presentinvention;

FIG. 15 is a schematic view of a layered image display interface of animage processing apparatus as depicted in FIG. 14;

FIG. 16 is a schematic view of an image characteristic editing interfaceof an image processing apparatus as depicted in FIG. 14;

FIG. 17 is a schematic view of a layered image editing interface of animage processing apparatus as depicted in FIG. 14;

FIG. 18 is a schematic view of a layered image display interface inaccordance with a preferred embodiment of the present invention;

FIG. 19 is a schematic view of a layer control panel in accordance withanother preferred embodiment of the present invention; and

FIG. 20 is a schematic view of a layer control panel in accordance witha further preferred embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In general, a human visual system is used for identifying an edge and aline of an image, which is an easy task for many occasions except for acamera system. Even though many relatively complicated theories andalgorithms have been adopted, it is not easy to simulate the recognitioncapability of a human visual system due to the following factors:

(1) An error caused by the quantization of the original image and adetected noise will cause an area of an edge or a line of the image todisappear as the detected brightness varies;(2) The precise positions of an edge and a line of an image may beaffected by quantized errors and noises to produce a deviation; and(3) An edge and a line of an object in an image should show the changeof a sharp brightness theoretically, due to the high-frequencycharacteristic, and thus any smooth filtered wave for reducing the noiseof an image will cause a blurring to the signals at the edge areas.Since a vast majority of edge detection methods adopt the derivativeprocessing, not only amplifying the high-frequency signals, but alsoamplifying the noises as well, therefore it is necessary to perform asmoothing filter to the signals, and the smoothing filter effect dependson the size of the filter and the scale of the filtered wave. The largerthe scale of filter, the broader is the range of brightness, but theprecision detected at the edge positions also becomes lower as well. Thesmaller the scale of filter, the more precise is the edge position, butit is easier to generate errors at the edge points. Furthermore, theinfluence on different scaled images will be different, and thus it isnot easy to appropriately find the best scale for all edges of an imagewith all edges, not even mentioning about finding an appropriate scaleof all images.

In the present image processing area, the generally acknowledge bestedge detection method is called a multi-scale edge detection, and themain concept of the method is to applied a smoothing filter (such as aGaussian filter) with different scales to perform a convolution with theoriginal image to obtain a filtered image with a different scale andthen take the edge of the filtered image for every scale, and finallystack all edges with all scales to form an edge image.

As an object can be expressed in different scales and displayed in animage. For instance, if a camera is shooting a photograph of a personwalking towards the camera and capturing the images continuously, thescale of a human face falls within a range of size from 3×3 pixels to300×300 pixels or even a larger range for a plurality of capturedimages. As the scale of the human face varies, a change ofcharacteristics of the human face will show up in the captured images,which is equivalent to the characteristics of the human face beingchanged continuously on the photographer's retina, so that thephotographer's visual cortex at a high level perception area senses thequantum jumps to form visual models with different scales of a differentgeneration.

Referring to FIGS. 10 and 11, the complexity of a graph structure willbe increased if the size of an image 600 is enlarged, so that thecharacteristics of the image 600 become more significant. If a differentgeneration model with a different scale can be created, then a series ofmodels can be used for defining perceptual model spaces. From theexperience of human perceptual sensing, the perceptual model spaces canbe defined into five major regimes:

(1) A texture regime 610: If a viewer views a person at a distance, theviewer cannot identify an image of the human face of that person easily,and thus the color of skin is generally identified to segment the humanface in an image;

(2) A PCA regime 620: This regime was proven as a regime capable ofshowing the characteristics of a scene of a mid-scale image the best;

(3) A parts regime 630: This regime has a higher resolution for clearlyidentify the image with five face features (including eyes, nose andmouths), and thus it can identify the movements of the five facefeatures (including the closing or opening movement of an eye or anose);

(4) A sketch regime 640: This regime has a much higher resolution, fordisplaying characteristics in more details to identify the five humanface features including eyelids, eyebrows, and crow's feet; and

(5) A super-resolution regime 650: This regime has a much higherresolution for displaying characteristics in more details.

From the description above, any image can generate a different visualmodel according to a different scale of its perceptual space. Thepresent invention adopts this concept together with the Gaussian andLaplacian pyramid theory as illustrated in FIG. 12 to convert anoriginal image into a plurality of scaled images 700, 710, 720, 730,740, 750 in a visual model with a different scale (as shown in FIG. 13,)and the image difference among scaled images 700, 710, 720, 730 of twoadjacent different scales is used for the computation, and a computedimage difference is used as a layered image 701, 711, 721, 731 of thecorresponding layer, such that each layered image 701, 711, 721, 731 canbe shown in a sequence of different levels from a clear level to a vaguelevel for displaying the characteristics of an edge and a line of theoriginal image. The present invention also provides a layered imagedisplay interface and an image characteristic editing interface, so thata user can examine each layered image 701, 711, 721, 731 in a visualmodel of a different scale of the original image through the layeredimage display interface, and edit or perform a special effect for eachlayered image 701, 711, 721, 731 through the image characteristicediting interface, so as to simulate different visual effects based ondifferent vision models and the user's requirements.

It is noteworthy to point out that the mathematical algorithm forcomputing the image difference among scaled images of two adjacentdifferent scales have been disclosed in many technical literatures andjournals. These algorithms vary according to actual needs andobjectives, but the basic algorithm generally adopts the visual model ofthe Gaussian and Laplacian pyramid theory to obtain the user's expectededge and line characteristics of an original image in a visual model ofa different scale. Since these algorithms and mathematical models arenot intended to be covered in the patent claims of the presentinvention, therefore they will not be described here.

In a preferred embodiment of the present invention as shown in FIG. 14,the image processing apparatus includes the following interfaces:

(1) A scaled image conversion interface 800: This interface as shown inFIG. 12 is provided for reading an original image and converting theoriginal image into a plurality of scaled images 700, 710, 720, 730,740, 750 in a visual model of a different scale. In FIG. 13, an imagedifference among scaled images 700, 710, 720, 730 of two adjacentdifferent scales is computed and used as a layered image 701, 711, 721,731 of a corresponding layer, such that each layered image 701, 711,721, 731 can be shown in a sequence of different levels from a clearlevel to a vague level for displaying the characteristics including anedge and a line of the original image;

(2) A layered image display interface 810: The layered image displayinterface 810 as shown in FIG. 15 comprises a plurality of layer controlpanels 811, each having a layer display window 812 and an editing startbutton 813, wherein the layer display window 812 is provided fordisplaying a layered image of a corresponding layer, such that a usercan examine a layered image in a visual model with a different scale inthe original image through the layer display window 812, and determiningwhether or not to edit or perform a special effect for each layeredimage; and the editing start button 813 is provided for starting anediting program that allows users to edit a layered image by the editingprogram displayed on the layer display window 812;

(3) An image characteristic editing interface 820: The imagecharacteristic editing interface 820 as shown in FIG. 16 comprises aplurality of image characteristic menus 821 for users to click to editand adjust the characteristics including contrast, highlight, midtone,shadow and white balance of the layered image of a started editingprogram;

(4) A layered image editing interface 830: The layered image editinginterface 830 as shown in FIG. 17 comprises a layered image editingwindow 831 for displaying each editing layered image or a composed imagecomposed of the layered images; at least one layered imagecharacteristic adjusting button 832 disposed at the periphery of thelayered image editing interface 830 for users to click to adjust thecontrast or Gaussian variance of the layered image; at least one sizeadjusting button 833 for users to click to adjust the process of zoomingin or out the layered image or the composed image; and at least oneimage composition switch button 834 for users to click to switch thelayered image editing window 831 to select and show each layered imageor composed image, such that a user can click the image compositionswitch button 834 to browse each editing layered image or composedimage; and

(5) An image output interface 840: The image output interface 840 isprovided for integrating each edited layered image into a new image asshown in FIG. 14 and converting the new image into an image file havinga specific format for saving, displaying and printing the image file.

Referring to FIGS. 15 and 17 for another preferred embodiment of thepresent invention, each layer control panel 811 further installs a layerdisplay button 814 for users to click to control whether or not todisplay a corresponding layered image in the layered image editingwindow 831. Therefore, users can click a layer display button 814 on thelayer control panel 811 according to actual requirements to select toopen or close the corresponding layered image in the layer control panel811, and click an editing start button 813 of the layer control panel811 to determine editing or performing a special effect for the layeredimages.

Referring to FIGS. 16 and 18 for another preferred embodiment of thepresent invention, each layer control panel 861 further adds a maskdisplay window 862 and a mask menu 863, wherein the mask menu 863 isprovided for users to click the desired mask and display the mask on themask display window 862, for displaying a mask on a corresponding layer,such that users can click a plurality of image characteristic menus 821in the image characteristic editing interface 820 to edit the mask.

Referring to FIGS. 17 and 19 for another preferred embodiment of thepresent invention, each layer control panel 871 adds a mask displaybutton 872 for users to click to control whether or not to display themask in the layered image editing window 831.

Referring to FIG. 16 for another preferred embodiment of the presentinvention, the image characteristic editing interface 820 further adds aprofile menu 822, for users to click, to determine whether or not toperform an access by the existing profile of the layered image.

In another preferred embodiment of the present invention as shown inFIG. 16, the image characteristic editing interface 820 adds a LMSchannel menu 823, for users to click to select a LMS channel.

In another preferred embodiment of the present invention as shown inFIG. 20, the layer control panel 881 adds a blurring or fine-tuneswitching button 882 for users to click to adjust the blurring orfine-tune of the pixels of the layered image.

In another preferred embodiment of the present invention as shown inFIG. 20, the layer control panel 881 adds a noise reduction button 883,for users to click to effectively prevent the interference of a noise tothe layered image.

It is noteworthy to point out that the editing and composition describedin the preferred embodiment is used for the illustration purposes only,and the persons skilled in the art should be able to use the Gaussianand Laplacian pyramid theory and the concept of the present invention toconvert an original image into a plurality of scaled images and displayeach layered image in visual models of different scales in the originalimage through a layered image display interface, so that users can editor perform a special effect to each layered image through an imagecharacteristic editing interface. Based on different vision models,different visual effects can be simulated. All mathematical conversionprograms or editing and composition programs of this sort are covered inthe scope of the patent claims of the present invention.

1. A method comprising: converting an original image into a plurality ofscaled images, each scaled image being at a predetermined scale withrespect to the original; generating from the plurality of scaled imagesa plurality of layered images, each layered image generated bycalculating the difference between two adjacent scaled images; providinga layered image display interface for displaying simultaneously to auser at least two layered images and providing in association with eachdisplayed layered image an edit start interface allowing the user toedit the layered image with an image characteristic editing interface;providing the image characteristic editing interface allowing the userto adjust a characteristic of the layered image thereby generating amodified layered image.
 2. The method of claim 1, further comprising:providing a layered image editing interface allowing the user to displayand edit a composed image generated in dependence upon at least themodified plurality of layered images and the original image.
 3. Themethod of claim 2, further comprising: providing an image outputinterface for generating in dependence upon the modified plurality oflayered images and the original image a new image, and converting saidnew image into an image file having a predetermined format.
 4. An imageprocessing apparatus comprising: a scaled image conversion interface,for receiving an original image, converting said original image into aplurality of scaled images, each scaled image at a predetermined scalewith respect to the original image, and generating from the plurality ofscaled images a plurality of layered images, each layered imagegenerated by calculating the difference between two adjacent scaledimages; a layered image display interface, including at least two layercontrol panels of a plurality of layer control panels, each layercontrol panel comprising a layer display window and an editing startbutton, wherein said layer display window is provided for displaying apredetermined layered image of the plurality of layered images allowinga user to examine the predetermined layered image of the plurality oflayered images and said editing start button for starting an imagecharacteristic editing interface for editing the corresponding layeredimage of the plurality of layered images; and the image characteristicediting interface, including a plurality of image characteristic menus,for editing and adjusting a characteristic value of said correspondinglayered image of the plurality of layered images to generate a modifiedlayered image and thereby generate a modified plurality of layeredimages.
 5. The apparatus of claim 4, further comprising a layered imageediting interface, and said layered image editing interface comprising:a layered image editing window, allowing the user to select and displaythe selected layered image of the modified plurality of layered imagesand a composed image generated in dependence upon at least the modifiedplurality of layered images and the original image; at least one layeredimage characteristic adjusting button for adjusting the contrast orGaussian variance of said layered image of the modified plurality oflayered images; at least one size adjusting button, for adjusting aprocess of zooming in or zooming out of said layered image of themodified plurality of layered images or said composed image; and atleast one image composition switch button, allowing the user to directsaid layered image editing window to selectively show the selectedlayered image of the modified plurality of layered images or thecomposed image.
 6. The apparatus of claim 4, further comprising an imageoutput interface for generating a new image in dependence upon themodified plurality of layered images and the original image, andconverting said new image into an image file with a predetermined formatfor at least one of saving, displaying and printing the image file. 7.The apparatus of claim 4, wherein said each layer control panel furthercomprises a layer display button, for controlling whether or not todisplay the predetermined layered image of the plurality of layeredimages on said layered image editing window.
 8. The apparatus of claim4, wherein said each layer control panel further comprises a maskdisplay window and a mask menu, said mask menu being provided for theuser to select and apply a mask of a plurality of masks to saidpredetermined layered image of the plurality of layered images and saidmask display window for displaying the result of applying said selectedmask to said corresponding predetermined layered image of the pluralityof layered images.
 9. The apparatus of claim 8, wherein said each layercontrol panel further comprises a mask display button, for controllingwhether or not to display said selected mask in said layer editingwindow.
 10. The apparatus of claim 4, wherein said each layer controlpanel further comprises at least one of a blurring and a fine-tuneswitching button, for adjusting a blurring and fine-tuning of pixels ofsaid predetermined layered image of the plurality of layered images. 11.The apparatus of claim 4, wherein said each layer control panel furthercomprises a noise adjustment button, for adjusting the effect of noiseupon the predetermined layered image of the plurality of layered images.12. The apparatus of claim 4, wherein said each image characteristicediting interface further comprises a profile menu allowing the user toa stored existing profile for use with the predetermined layered imageof the plurality of layered images.
 13. The apparatus of claim 4,wherein said each image characteristic editing interface furthercomprises a menu relating to a LMS color space representing theresponses of the three human retinal cones, the menu for selecting avariation of the LMS color space to be applied to the predeterminedlayered image of the plurality of layered images.